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METHOD OF CALCULATING NOISE FROM MULTIPLE DIGITAL 
IMAGES UTILIZING COMMON NOISE CHARACTERISTICS 
FIELD OF INVENTION 

The present invention relates to a method for enhancing digital 
5 images using a noise characteristic common to such noise characteristics. 

BACKGROUND OF THE INVENTION 
Some digital image processing applications designed to enhance 
the appearance of the processed digital images take explicit advantage of the noise 
characteristics associated with the source digital images. For example, Keyes et 
10 al. in U.S. Patent 6,1 18,906 describe a method of sharpening digital images which 
includes the steps of measuring the noise components in the digital image with a 
noise estimation system to generate noise estimates and sharpening the digital 
image with an image sharpening system which uses the noise estimates. 
Similarly, digital imaging applications have incorporated automatic noise 
15 estimation methods for the purpose of reducing the noise in the processed digital 
images as in the method described by Anderson et al. in U.S. Patent 5,809,178. 

In U.S. Patent 5,923,775, Snyder et al. disclose a method of image 
processing which includes a step of estimating the noise characteristics of a digital 
image and using the estimates of the noise characteristics in conjunction with a 
20 noise removal system to reduce the amount of noise in the digital image. The 
method described by Snyder et al. is designed to work for individual digital 
images and includes a multiple step process for the noise characteristics estimation 
procedure. First, the residual signal is formed from the digital image obtained by 
applying a spatial filter to the digital image. This first residual is analyzed to form 
25 a mask signal which determines what regions of the digital image are more and 
less likely to contain image structure content. The last step includes forming a 
second residual signal and sampling the residual in image regions unlikely to 
contain image structure content to form the noise characteristic estimation. 

In U.S. Patent 6,069,982, Reuman describes a method of estimating 
30 the noise characteristics of a digital image acquisition device which includes 
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providing predetermined default values for the spatial noise characteristics of the 
digital image acquisition device, gathering information related to the spatial noise 
characteristics of the digital image acquisition device; generating replacement data 
in response to the gathered information; and updating the predetermined default 

5 spatial noise characteristics associated with the digital image acquisition device 
with the replacement data. In particular the method disclosed by Reuman 
estimates the standard deviation (derived from the variance) as a function of the 
grey-level (pixel value) and the spatial frequency characteristics of the noise. The 
noise characteristics, such as a table of standard deviation values as a function of 

10 grey-level, are provided as the default values. Each digital image to be processed 
is analyzed which includes the calculation of statistical quantities in the gathering 
of information step. These statistical quantities and the default values are 
combined to calculate the updated replacement noise characteristic values. 
Furthermore, in the method taught by Reuman, a yes/no decision is made with 

15 regard to the presence or the lack of the default noise values. If the default noise 
values are present, they are used. If the default noise values are not present, they 
are generated from the digital image to be processed with the aid of the operator of 
the system. 

In the method taught by Reuman, the estimated noise characteristic 
20 values are specific to a digital image acquisition device. Reuman gives examples 
of digital images acquisition devices such as a film scanner, digital camera, or an 
image processing module. Although film scanners can add noise to the digital 
images they produce, much of the noise present in the output digital images can be 
attributed to the photographic film type. Thus, the noise characteristic values 
25 associated with the digital images derived from a scanned photographic film can 
be attributed to the type of photographic film independent of the film scanner 
device used to produce the digital image. 

In U.S. Patent 5,959,720 Kwon et al. disclose a method of color 
balance determination for use by a color copying apparatus utilizing multiple 
30 image frames of a photographic film order. Multiple individual digital images are 



produced by a film scanner and the pixel data of these digital images is 
collectively analyzed to calculation a color balance point relating to the set of 
digital images. The color balance point is then used to process each digital image 
for improved color balance. The method disclosed by Kwon et al. is an example 
5 of a digital imaging analysis method which combines the analysis of pixel data 
from multiple digital images to improve the processing of pixel data for each of 
the digital images. 

SUMMARY OF THE INVENTION 

It is an object of the present invention to provide an improved way 
10 of enhancing digital images, which are believed to be affected by a common noise 
source. 

This object is achieved by a method of enhancing one or more 
digital images from a plurality of digital images that are believed to be affected by 
a common noise source, comprising the steps of: 

15 a) receiving two or more source digital images that are 

believed to be affected by a common noise source; 

b) using the pixels of the received source digital images to 
calculate a noise characteristic value that relates to the noise present in the 
received source digital images; and 

20 c) using the noise characteristic value and the received source 

digital images to respectively calculate enhanced digital images for the one or 
more of the received source digital images. 

It is a feature of the present invention to provide for improved 
image enhancement by taking advantage of the fact that for a particular group of 

25 digital images, they will contain similar amounts of noise. The present invention 
is particularly advantageous for enhancing images which are taken on a common 
photographic film type. These images will all share common noise characteristics 
which are substantially contributed to by the photographic film. Nevertheless, the 
present invention also is useful when the digital images are provided by a scanner 

30 or digital camera which also introduces common noise. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

FIG. 1 is a functional block diagram showing the component parts 
of a digital imaging system implementation of the present invention; 

FIG. 2 is a functional block diagram of the digital image processor 
5 module shown in FIG. 1 employed by a preferred embodiment of the present 
invention; 

FIG. 3 is a function block diagram of the noise estimation module 
shown in FIG. 2 used by the preferred embodiment of the present invention; 

FIG. 4 is a function block diagram of the noise estimation module 
10 used by an alternative embodiment of the present invention; and 

FIG. 5 is a function block diagram of the noise estimation module 
used by another alternative embodiment of the present invention. 

DETAILED DESCRIPTION OF THE INVENTION 
In the following description, a preferred embodiment of the present 
15 invention will be described as a software program. Those skilled in the art will 
readily recognize that the equivalent of such software may also be constructed in 
hardware. Because image manipulation algorithms and systems are well known, 
the present disclosure will be directed in particular to algorithms and systems 
forming part of, or cooperating more directly with, the method in accordance with 
20 the present invention. Other aspects of such algorithms and systems, and 
hardware and/or software for producing and otherwise processing the image 
signals involved therewith, not specifically shown or described herein may be 
selected from such systems, algorithms, components, and elements known in the 
art. Given the description as set forth in the following specification, all software 
25 implementation thereof is conventional and within the ordinary skill in such arts. 

The present invention may be implemented in computer hardware. 
Referring to FIG. 1 , the following description relates to a digital imaging system 
which includes an image capture device 10a, a digital image processor 20, an 
image output device 30a, and a general control computer 40. The system may 
30 include a monitor device 50 such as a computer console or paper printer. The 
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system may also include an input device control 60 for an operator, such as a 
keyboard and or mouse pointer. Multiple capture devices 10a, 10b, and 10c are 
shown illustrating that the present invention may be used for digital images 
derived from a variety of imaging devices. For example, FIG. 1 may represent a 
5 digital photofinishing system where the image capture device 10a is a 

conventional photographic film camera for capturing a scene on color negative or 
reversal film, and a photographic film scanner for scanning the developed image 
on the film and producing a digital image. Although the term scanner can refer to 
digital imaging devices that physically scan or move a sensing element past a 

10 photographic film sample, the present invention also includes photographic film 
scanners and print scanners that employ a stationary image sensing device to 
generate a digital image. The digital image processor 20 the receives a set of 
digital images, processes these digital images to produce an enhanced digital 
image for one or more digital image in preparation for the intended output device 

15 or media. The digital image processor 20 analyzes the noise characteristics of the 
set of digital images and modifies the spatial characteristics of these digital images 
by employing a noise reduction filter and a spatial sharpening filter. In addition, 
the digital image processor 20 may process the set of digital images to make 
adjustments for color, density and tone scale in a manner such that a pleasing 

20 looking image is produced by an image output device 30a. Those skilled in the art 
will recognize that the present invention is not limited to just these mentioned 
image processing modules. Multiple image output devices 30a and 30b are shown 
illustrating that the present invention may be used in conjunction with a variety of 
output devices which may include a digital photographic printer and soft copy 

25 display. The present invention uses a digital photographic printer to make a print 
of the enhanced digital images to make photographic prints. 

The general control computer 40 shown in FIG. 1 may store the 
present invention as a computer program stored in a computer readable storage 
medium, which may comprise, for example: magnetic storage media such as a 

30 magnetic disk (such as a floppy disk) or magnetic tape; optical storage media such 
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as an optical disc, optical tape, or machine readable bar code; solid state electronic 
storage devices such as random access memory (RAM), or read only memory 
(ROM). The associated computer program implementation of the present 
invention may also be stored on any other physical device or medium employed to 

5 store a computer program indicated by offline memory device 70. Before 
describing the present invention, it facilitates understanding to note that the 
present invention is preferably utilized on any well known computer system, such 
as a personal computer. 

It should also be noted that the present invention implemented in a 

10 combination of software and/or hardware is not limited to devices which are 
physically connected and/or located within the same physical location. One or 
more of the devices illustrated in FIG. 1 may be located remotely and may be 
connected via a wireless connection. 

A digital image is comprised of one or more digital image 

15 channels. Each digital image channel is comprised of a two-dimensional array of 
pixels. Each pixel value relates to the amount of light received by a image capture 
device corresponding to the geometrical domain of the pixel. For color imaging 
applications a digital image will typically consist of red, green, and blue digital 
image channels. Other configurations are also practiced, e.g. cyan, magenta, and 

20 yellow digital image channels. For monochrome applications, the digital image 
consists of one digital image channel. Motion imaging applications can be 
thought of as a time sequence of digital images. Those skilled in the art will 
recognize that the present invention can be applied to, but is not limited to, a 
digital image channel for any of the above mentioned applications. Although the 

25 present invention describes a digital image channel as a two dimensional array of 
pixel values arranged by rows and columns, those skilled in the art will recognize 
that the present invention can be applied to mosaic (non rectilinear) arrays with 
equal effect. 

While the discussion of the present invention hereinbelow 
30 describes processing digital images, it should be noted that processing digital 



7 



images does not necessarily require that the entirety of a digital image must be 
processed. For example, selected regions within a digital image can be processed 
without processing the entire digital image and the digital image can still be said 
to have been processed. Similarly, the present invention can be practiced by 
5 processing only some of the digital image channels included in a digital image. It 
should also be noted that the present invention can be practiced with digital 
images of a variety of different image representations. For example, the present 
invention can be practiced with digital images having color pixels of red, green 
and blue. However, the present invention can also be practiced with digital 

10 images in a luminance-chrominance image representation. 

The digital image processor 20 shown in FIG. 1 is illustrated in 
more detail in FIG. 2. The general form of the digital image processor 20 
employed by the present invention is a cascaded chain of image processing 
modules. The noise estimation module 110 receives the source digital images 101 

15 and calculates a local noise characteristic table 105, i.e. a table of noise 

characteristic values, using the pixel values from the source digital images 101. 
Each image processing module contained within the digital image processor 20 
receives a digital image, modifies the digital image, produces a processed digital 
image and passes the processed digital image to the next image processing 

20 module. The two enhancement transform modules shown within the digital image 
processor 20 are a noise reduction module 22 and a spatial sharpening module 23. 
These two modules use the local noise characteristic table 105 produced by the 
noise estimation module 1 10 to generate the enhanced digital images 102. Those 
skilled in the art will recognize that the any other image processing module that 

25 utilizes a noise characteristic table can be used with the present invention. 

The noise estimation module 110 shown in FIG. 2 is illustrated in 
more detail in FIG. 3. The source digital images 101 are received by the digital 
image indexer 115 which dispatches each source digital image 101 to the residual 
transform module 120 for processing. The residual transform module 120 

30 receives a source digital image 101, performs a spatial filtering operation on the 



pixel data of the source digital image 101 resulting in a residual digital image. 
The residual digital image produced for each source digital image 1 0 1 is received 
by the residual statistical accumulator 130 which calculates a set of local residual 
histograms. When the digital image index er 115 has dispatched all of the source 
5 digital images 101, the digital image indexer 115 sends a message to the noise 
table calculator 140 that all of the source digital images 101 have been processed. 
The noise table calculator 140 receives the set of local residual histograms and 
produces a local noise characteristic table 105. 

The residual transform module 120 uses a residual spatial filter to 

10 perform a spatial filtering operation on the pixel data of a digital image. A 

residual pixel value is generated for each original pixel value in the source digital 
image 101 by the residual spatial filter. For each pixel of interest, a combination 
of pixel values sampled from a local region of pixels is used to form the residual 
pixel value. If the source digital image 101 is a color digital image, the residual 

15 transform module 120 performs the spatial filtering operation on each color digital 
image channel and forms a residual pixel value for each pixel of each color digital 
image channel. The preferred embodiment of the present invention uses a two- 
dimensional Laplacian operator as the spatial filter to form the residual pixel 
values. The Laplacian operator calculates a local arithmetic mean value from the 

20 value of pixel sampled from the local region of pixels about the pixel of interest 
and subtracts the value of the pixel of interest from the local arithmetic mean 
value. A local region of 3 by 3 pixels is used. The Laplacian operator is a 
convolution spatial filter with an associated convolution kernel of: 



1 1 1 
1 -1 1 
1 1 1 



(1) 



Although the preferred embodiment of the present invention uses a two 
30 dimensional Laplacian based residual spatial filter, those skilled in the art will 
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recognize that the present invention can be practiced with other spatial filters, such 
as but not limited to, one-dimensional Laplacian spatial filters. 

An alternative embodiment of the present invention uses the 
method disclosed by Snyder et al. in U.S. Patent 5,923,775. In this alternative 
5 embodiment, a similar technique of forming a residual pixel value is performed. 
Next, a gradient signal is calculated using a spatial filter. The gradient signal is 
analyzed forming a masking signal that is used to reject some of the residual pixel 
values from later consideration. Although this alternative embodiment leads to 
more accurate noise estimation it is also more computationally intensive than the 

10 preferred embodiment. 

The pixel data of the source digital image 101 can be 
conceptualized as having two components - a signal component relating to 
photographed objects and a noise component. The resulting residual pixel values 
have statistical properties that have a closer relationship to the noise component of 

15 the pixel data of the source digital image 101 than the signal component. 

Although the noise component can contain sub-components, the stochastic sub- 
component of the noise component is well modeled by a zero mean Gaussian 
probability distribution function. To first order, the noise component of the pixel 
data of the source digital image 101 can be characterized by a standard deviation 

20 and a mean value of zero. To second order, standard deviation of the noise 
component can be modeled as being signal and color channel dependent. 

The residual transform module 120 analyzes the residual pixel 
values and records these values in the form of a set of local residual histograms as 
a function of the pixel color and numerical pixel value. Therefore, a given local 

25 residual histogram H lk relates to the i th color digital image channel and the k th pixel 
value sub-range. For each pixel of interest denoted by p mn (corresponding to the 
m th row and n* column location) in the processed color digital image channel, a 
histogram bin index k is computed. For example, if the numerical range of pixel 
values is from 0 to 255 there can be as many as 256 useful histograms, i.e. one 

30 histogram for each possible numerical pixel value. In general, most noise sources 
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can be characterized as having noise standard deviations that are slow functions of 
the pixel value. Therefore, the preferred embodiment of the present invention uses 
8 histograms to cover the numerical pixel value range from 0 to 255. Thus, the 
calculated histogram index bin and the corresponding sub -range pixel values are 
5 given by the following Table 1 . 



Table 1 



histogram bin index 


sub-range pixel values 


average pixel value 


0 


0to31 


16 


1 


32 to 63 


48 


2 


64 to 95 


80 


3 


96 to 127 


112 


4 


128 to 159 


144 


5 


160 to 191 


176 


6 


192 to 233 


208 


7 


234 to 255 


240 



10 Those skilled in the art will recognize that the present invention can be practiced 
with digital image pixel data with any numerical range. The number of local 
residual histograms used for each color digital image channel will depend on the 
accuracy of results required for the particular digital imaging application. 

Although each local residual histogram records statistical 

15 information for a range of pixel values for a given color digital image channel, the 
local residual histogram records the frequency of residual pixel values associated 
with each pixel of interest p mn . Since the expected mean of the distribution of 
residual pixel values is zero, the residual pixel values exhibit both positive and 
negative values. Therefore, the local residual histogram must record the 

20 frequency, i.e. the number of instances of residual pixel values, of all possible 
instances of residual pixel values. For the example above, the residual pixel 



11 

values can range from -255 to +255. While it is possible to construct local 
residual histograms with as many recording bins as there are possible instances of 
residual pixel values, in general it is not necessary. For most digital images, only 
a small percentage of residual pixel values exhibit values near the extremes of the 
5 possible range. The present invention uses 101 total recording bins for each local 
residual histogram. One of the recording bins corresponds to residual pixel values 
of 50 and greater. Similarly, one other recording bin corresponds to residual pixel 
values of -50 and lower. The other 99 recording bins each correspond to a single 
residual pixel value for the numerical range from -49 to +49. 
10 Referring to FIG. 3, the noise table calculator 140 receives a set of 

local residual histograms and calculates the local noise characteristic table 105 in 
the form of a table of standard deviation values. For each of the local residual 
histograms relating to a particular color digital image channel and pixel value 
range, the noise table calculator 140 derives a noise standard deviation value from 
15 the value of the recording cells of the local residual histogram. The preferred 
embodiment of the present invention uses equation (2) to calculate the standard 
deviation value c n 

ct„ = ((1/N) E k RC v (k) (x-x ni ) 2 ) m (2) 
where the variable x represents the average pixel value of the residual pixel values 
accumulated in the k t!l recording cell as given by Table (1) and RCv(k) represents 
the number of residual pixel values accumulated by the k th recording cell. 

x = V(k) (3) 
The variable x m represents the arithmetic mean value of the corresponding residual 
pixel values given by equation (4), 

x m = (1/N) 2 k x (4) 
and the variable N represents the total number of residual pixel values recorded by 
the updated residual histogram given by equation (5). 

N = L k RC v (k) (5) 



20 



25 
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An alternative embodiment of the present invention performs an 
alpha-trimmed standard deviation calculation. In this embodiment a first 
approximation to the standard deviation a e is calculated using the method 
described above. The calculation of a n is then calculated using the only recording 
5 cells with corresponding residual pixel values that are within a limited range of 
zero. The formula for the standard deviation calculation <r n is given by 
equation (6) 

o„ = ((1/N) S kY RC v (k) (x-x m ) 2 ) 1/2 (6) 
where the variable y is given by equation (7) 

10 y = 1 if j x | < a cr e (7) 

y = 0 if | x | >= a <r e 
where the variable a is set to 3.0. This alternative embodiment of the present 
invention is more computationally intensive than the preferred embodiment but 
does yield more accurate results via the rejection of out-lying residual pixel values 
15 from adversely contributing to the calculation of the standard deviation a n value. 

Table 2 below is an example of a noise characteristic table 
produced with the present invention. 

Table 2 

20 



average pixel 
value 


Standard 
deviation of red 
channel 


Standard 
deviation of 
green channel 


Standard 
deviation of 
blue channel 


16 


3.28 


3.62 • 


3.21 


48 


3.71 


3.20 


3.38 


80 


3.77 


4.14 


4.50 


112 


4.57 


4.35 


4.21 


144 


4.98 


4.25 


4.37 


176 


5.05 


4.11 


6.21 


208 


5.05 


5.64 


6.29 


240 


2.71 


4.27 


3.87 
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Those skilled in the art will recognize that the present invention can be practiced 
with calculated quantities other than the standard deviation that relate to the noise 
present in digital images. For example, the statistical variance (a squared function 
5 of the standard deviation) or statistical median can also be derived from the 
residual histograms and be used to form a table of noise characteristic values. 

The present invention uses a set of residual histograms to record 
the calculated statistics. A set of histograms is an example of a statistical table 
from which a noise characteristic table can be derived. Thus, the set of local 

10 residual histograms constitutes a statistical table, i.e. a local statistical table. 

Those skilled in the art will recognize that the present invention can be practiced 
with other forms of statistical tables. For example, the residual digital images 
could be stored and serve as a statistical table. It should be noted that the present 
invention uses a set of residual histograms as the form for the statistical table due 

15 to its inherent computational and storage simplicity and economy. 

The present invention produces more accurate noise characteristic 
tables through the feature of combining the residual statistics of multiple digital 
images. With more pixel data considered from multiple digital images, the 
standard deviation values of the calculated noise characteristic table converge to 

20 the true inherent noise characteristics of the underlying recording medium. For 
many digital imaging applications, a plurality of digital images derived from a 
common image source will be affected by a common noise source. 

The above discussion has included details of practicing the present 
invention for digital images of general type. However, most digital imaging 

25 systems accept digital images from a variety of sources. For example, the image 
capture device 10a and 10b shown in FIG. 1 could be a photographic film scanner 
while the image capture device 10c could be a digital camera, digital camcorder, 
or print scanner. The image capture device can contribute noise to the digital 
images it produces. However, the inherent noise in the capture medium usually 

30 dominates the overall noise characteristics of the resultant digital images. For 
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example, while a photographic film scanner can produce digital images from any 
photographic film type, in general, some photographic films are inherently noisier 
that others. A photographic film sample is an example of a photographic image. 
Other examples of photographic images can include, but are not limited to, a CCD 
5 imaging electronic device and a photographic print. 

In an alternative embodiment of the present invention, the image 
capture devices 10a, 10b, and 10c shown in FIG. 1 are capable of producing a 
source type identification tag 103, as shown in FIG. 2, which uniquely identifies a 
set of digital images as belonging to a particular group. In the example given 

10 above, a photographic film sample Kodak Generation 6 Gold 200 film is scanned 
by the image capture device 10a which produces a set of source digital images 101 
and a source type identification tag 103. The digital imaging system maintains a 
plurality of stored source type identification tags which correspond to a plurality 
of stored default noise characteristic tables which correspond to different types of 

15 photographic films, print scanners, and digital cameras or the like. The digital 
imaging system uses the source type identification tag to select the appropriate 
default noise characteristic table. Referring to FIG. 3, the source type 
identification tag 1 03 is received by the digital image indexer 1 1 5 of the noise 
estimation module 1 10 shown in FIG. 2. The source type identification tag 103 

20 identifies the source digital images 101 as being Kodak Generation 6 Gold 200 

film. Digital images produced by other image capture devices, such as a particular 
model digital camera, have a corresponding unique source type identification tag. 
In similar fashion, the set of digital images marked with a source type 
identification tag are processed to produce a local noise characteristic table 105 

25 which in turn is used to generate enhanced digital images 102. The source type 
identification tag 103 can also further identify a set of digital images that belong 
to a particular consumer. For this alternative embodiment, just the digital images 
marked with the particular consumer's source type identification tag 103 will be 
processed as a set. Of course, it is also possible that a particular consumer may 

30 issue more than one set of photographic images for processing. 
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The above described alternative embodiment of the present 
invention makes use of grouping digital images based on the source of the 
imagery. The statistical accuracy of the calculated noise characteristic table can 
be further improved if the statistics from previously processed sets of source 
5 digital images derived from the same source of imagery are combined. In a 

further alternative embodiment of the present invention a default statistical table 
106 (as shown in FIG. 4), i.e. maintained and provided by the digital imaging 
system, is used in conjunction with the pixels from the source digital images 101 
to calculate a local noise characteristic table 105 for the source digital images 101. 

10 However, the digital imaging system, shown in FIG. 1, stores a default statistical 
table 106 corresponding to each unique source type identification tag 103. Thus, 
the default statistical table 106 corresponding to Kodak Generation 6 Gold 200 
film is used to process the source digital images 101 derived from the scanning the 
Kodak Generation 6 Gold 200 film sample with the image capture device 10a. It 

1 5 is important to note that if a different sample of Kodak Generation 6 Gold 200 
film is scanned by the image capture device 10b, the same default statistical table 
corresponding to Kodak Generation 6 Gold 200 film is used. Thus, the default 
statistical table 106 is selected on the basis of the type of photographic film and 
not necessarily on the type of or individual unit image capture device. This 

20 feature of the present invention allows the default statistical table 106, and 

consequently the resultant calculated local noise characteristic table 105, to track 
or relate to the type of photographic film manufactured. Since the present 
invention automatically updates the default statistical table 106 and can derive the 
default statistical table 106 from the pixel values of digital images, the present 

25 invention can be used with new types of manufactured photographic film without 
the need of a disseminated a data base of either statistical tables or noise 
characteristic tables. 

Those skilled in the art will recognized that this feature of the 
present invention can easily be extended to include other sources of digital 

30 images. For example, the image capture device 10c can be a digital still camera, 
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such as the Kodak DCS 290. For this example, the image capture device 10c 
produces a unique source type identification tag 103. In this manner any newly 
produced digital camera which produces a new and unique source type 
identification tag can be processed effectively with the present invention. When 

5 the digital imaging systems shown in FIG. 1 encounters a previously unknown 
source type identification tag 1 03, a new default statistical table 106 is created. 

In another alternative embodiment of the present invention, the 
digital imaging system shown in FIG. 1 maintains a separate database of default 
statistical tables 106, one for each source identification tag 103, for each image 

10 capture device 10a and 10b. Since the image capture device can contribute some 
noise to the digital images it produces, maintaining separate databases of default 
statistical tables for each image capture device results in more accurate noise 
characteristic tables. 

As shown in FIG. 4, the residual statistical accumulator 130 

15 combines the default statistical table 106, in the form of a default residual 
histograms, with the local residual histograms. While the local residual 
histograms record residual pixel values as derived from the currently processed set 
of source digital images 101, the default residual histograms record the residual 
pixel values as derived from the previously processed digital images. Thus, the 

20 default residual histograms have exactly the same form as the local residual 

histograms, i.e. one local residual histogram for each pixel value sub-range of each 
color digital image channel. 

The present invention uses more than one method of combining the 
local residual histograms with the default residual histograms, however, each 

25 method combines a single local residual histogram with the corresponding default 
residual histogram. Therefore, it is appropriate only to discuss the method with 
respect to the combination of two histograms with the assumption that each 
combining method is repeated for all the pairs of histograms. 

This alternative embodiment of the present invention uses a direct 

30 method of combining the data contained in the local residual histogram and 
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default residual histogram. That is, the numbers contained in each recording cell 
of the local residual histogram are added directly to the corresponding recording 
cell of the default residual histogram. Thus, after the combining step has been 
performed, the updated values of each recording cell is given by the sum of the 
5 previous value of the recording cell with the value contained in the corresponding 
recording cell of the local residual histogram. 

Since the recording cells of the default residual histogram contains 
the sum total of residual pixel values derived from the set of source digital images 
as well as previous processed digital images, the numerical range of the computer 

10 implementation can be exceeded. To avoid a numerical overflow condition, the 
default residual histogram can be re-normalized. The process of re-normalization 
includes scanning the values of the recording cells to determine the maximum 
value, comparing this maximum value to a predetermined allowable value. If the 
maximum value is greater than the a predetermined allowable value, the values of 

15 all the recording cells are divided by a constant numerical factor. This process of 
re-normalization of the default residual histogram can be performed either before 
or after the process of combining the local residual histogram with the default 
residual histogram. The preferred embodiment of the present invention performs 
the re-normalization process before the combining process. 

20 An alternative embodiment of the present invention produces the 

updated residual histogram by calculating a weighted average between the default 
residual histogram and the local residual histogram. For each recording cell of the 
default residual histogram RC d and its corresponding recording cell of the local 
residual histogram RQ , the updated recording cell value RC V is calculated using 

25 the formula: 

RC V = p RC d + (1 - (3) RC, (8) 

where the variable p is a numerical weighting factor. The present invention uses p 
value of 0.99 for a which heavily weights the contribution of the default residual 
histogram. This alternative embodiment of the present invention uses a linear 
30 combination of the local residual histogram and the default residual histogram. 



The present invention can be practiced by recording a default noise 
characteristic table, i.e. maintained and provided by the digital imaging system, 
instead of recording the residual histograms. FIG. 5 shows the details of an 
alternative embodiment of the noise estimation module 110. As described in the 
5 alternative embodiment above, a set of local residual histograms is calculated from 
the source digital images 101. The noise table calculator 140 receives the local 
residual histograms and produces a local noise characteristic table 105. The noise 
table generator 150 receives the local noise characteristic table and a default noise 
characteristic table 107 and produces an updated noise characteristic table 108. 

10 This updated noise characteristic table 108 replaces the default noise characteristic 
table 107 for the next source digital image 101 to be processed. The local noise 
characteristic table 105 and default noise characteristic table 107 are combined by 
calculating a linear combination of these two tables element for element. Good 
values for the linear combination coefficients are 0.99 contribution for the default 

15 noise characteristic table values and 0.01 for the local noise characteristic table 

values. Although this alternative embodiment of the present invention can be used 
to generate robust noise characteristic tables, the alternative embodiment that 
combines residual histograms generally produces more accurate results. 

The calculated noise characteristic table is used in conjunction with 

20 spatial filters to produce an enhanced digital image 102 from a source digital 

image 101. A spatial filter is any method which uses pixel values sampled from a 
local region about a pixel of interest to calculate an enhanced pixel value which 
replaces the pixel of interest. Those spatial filters which reduce spatial 
modulation, for at least some pixels in an effort to remove noise from the 

25 processed digital image, can be considered noise reduction filters. Those spatial 
filters which increase spatial modulation, for at least some pixels, in an effort to 
enhance spatial detail noise in the processed digital image, can be considered 
spatial sharpening filters. It should be noted that it is possible for a single spatial 
filter to be considered both a noise reduction filter as well as a spatial sharpening 

30 filter. The present invention can be used with any digital image processing 



method which makes uses of a noise characteristic table to produce an enhanced 
digital image 102. Spatial filters that adjust a processing control parameter as a 
function of either the color or numerical value of pixels are adaptive spatial filters. 
The present invention uses a noise reduction filter and a spatial sharpening filter 
which are responsive to a noise characteristic table. 

Referring to FIG. 2, the preferred embodiment of the present 
invention employs a noise reduction module 22 as part of the image processing 
method to produce enhanced digital images 102. As such, the source digital 
image 101 and the local noise characteristic table 105 are received by the noise 
reduction module 22 which produces on output a noise reduced digital image. 

It is important to note that for many practical digital imaging image 
systems, other image processing processors need to be included. As long as these 
other image processing processors accept a digital image as input and produce a 
digital image on output, more of these type of image processing modules can be 
inserted in the image processing chain in between a noise reduction module 22 
and a spatial sharpening module 23. 

The present invention uses a modified implementation of the Sigma 
filter, described by Jong-Sen Lee in the journal article Digital Image Smoothing 
and the Sigma Filter, Computer Vision, Graphics, and Image Processing Vol 24, 
p. 255-269, 1983, as a noise reduction filter to enhance the appearance of the 
processed digital image. The values of the pixels contained in a sampled local 
region, n by n pixels where n denotes the length of pixels in either the row or 
column direction, are compared with the value of the center pixel, or pixel of 
interest. Each pixel in the sampled local region is given a weighting factor of one 
or zero based on the absolute difference between the value of the pixel of interest 
and the local region pixel value. If the absolute value of the pixel value difference 
is less or equal to a threshold b, the weighting factor if set to one. Otherwise, the 
weighting factor is set to zero. The numerical constant s is set to two times the 
expected noise standard deviation. Mathematically the expression for the 
calculation of the noise reduced pixel value is given as 
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q mn = S,j a„ P|J a, (9) 

and 

a,j = 1 if IPy-Pm,, ! <= s 

a y = 0 if |p„-p mn |> s 

5 where p y represents the ij th pixel contained in the sampled local region, p mn 
represents the value of the pixel of interest located at row m and column n, a y 
represents a weighting factor, and q m represents the noise reduced pixel value. 
Typically, a rectangular sampling region centered about the center pixel is used 
with the indices i and j varied to sample the local pixel values. 
10 The signal dependent noise feature is incorporated into the 

expression for s given by equation (10) 

e= Sfaconipj) (10) 

where a n represents the noise standard deviation of the source digital image 101 
evaluated at the center pixel value p mn as described by equations (3) and (8) above. 

15 The parameter Sfac is termed a scale factor can be used to vary the degree of noise 
reduction. The optimal value for the Sfac parameter has been found to be 1.5 
through experimentation however values ranging from 1.0 to 3.0 can also produce 
acceptable results. The calculation of the noise reduced pixel value q rail as the 
division of the two sums is then calculated. The process is completed for some or 

20 all of the pixels contained in the digital image channel and for some or all the 
digital image channels contained in the digital image. The noise reduced pixel 
values constitute the noise reduced digital image. The modified implementation 
of the Sigma filter is an example of a noise reduction filter that uses a noise 
characteristic table and is therefore an adaptive noise reduction filter which varies 

25 the amount of noise removed as a function of the pixel color and numerical value. 

Referring to FIG. 2, the preferred embodiment of the present 
invention employs a spatial sharpening module 23 as part of the image processing 
method to produce an enhanced digital image 102. As such, the noise reduced 
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digital image and the local noise characteristic table 105 are received by the spatial 
sharpening module 23 which produces on output an enhanced digital image 105. 

Although the present invention can be used with any spatial 
sharpening filter which utilizes a priori knowledge of the noise characteristics, the 
5 preferred embodiment uses a modified implementation of the method described by 
Kwon et al. in U.S. Patent 5,081,692. This spatial sharpening method performs an 
unsharp masking operation by filtering the input digital image with a spatial 
averaging 2-dimensional Gaussian filter (characterized by a standard deviation of 
2.0 pixels) which results in a blurred digital image. The blurred digital image is 

10 subtracted from the input digital image to form a high-pass residual. In the 
method disclosed by Kwon et al. a local variance about a pixel of interest is 
calculated by using the pixel data from the high-pass residual. Based on the value 
of the local variance a sharpening factor is adjusted so as to amplify large signals 
more than small amplitude signals. The amplification factor § is therefore a factor 

15 of the local variance v. i.e. (j>(v). 

The present invention modifies the method taught by Kwon et al. to 
make the amplification factor (j)(v) a function of the estimated noise, i.e. (|>(v,a n ). 
The amplification function f is given by a gamma function, or integral of a 
Gaussian probability function, as given by equation (11). 



where y 0 represents a minimum amplification factor y max represents a maximum 
amplification factor, v max represents a maximum abscissa value of the variable v, 
v 0 represents a transition parameter and s represents a transition rate parameter. 
The variable v 0 is a function of the noise standard deviation value a n as per 
25 equation (12) 

v Q = Sfac 2 <yAVnJ (12) 
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where the scaling factor Sfac2 determines the sensitivity of the sharpening 
sensitivity to the noise and the noise standard deviation value a n is as described 
above in equations (3) and (8). The optimal values for the variables used in 
equation (12) depend on the digital imaging application. The present invention 
5 uses a value of 1 .0 for y 0 which results in no spatial sharpening for noisy regions. 
A value of 3.0 is used for y max , however, this variable is sensitive to user 
preference with values ranging from 2.0 to 4.0 producing acceptable results. The 
value of Sfac2 should be set to between 1 .0 and 2.0 with 1 .5 as optimal. The 
variables should be set to values in the range from vo / 2 to vo / 10 for reasonable 

10 results. The variable v max should be set to a value much larger than the expected 
noise, e.g. 20 times the value of o n . 

While the preferred embodiment of the present invention calculates 
a noise characteristic table and then subsequently uses the noise characteristic 
table to produce an enhanced digital image, some digital imaging systems may be 

15 configured to separate the calculation phase from the enhancement phase. In an 
alternative embodiment of the present invention, the calculated noise characteristic 
table is stored with the source digital image 101 as meta-data, i.e. non-pixel 
information. The source digital image 101 with meta-data can be transmitted to a 
remote site or stored for safe keeping to be used at a later time or another site. 

20 Any of the above mentioned noise characteristic tables can be stored as meta-data. 
In general a noise characteristic table requires much less memory storage than a 
set of residual histograms. However, a set of residual histograms can be stored 
with the source digital image 101 as meta-data. 

A computer program product may include one or more storage 

25 medium, for example; magnetic storage media such as magnetic disk (such as a 
floppy disk) or magnetic tape; optical storage media such as optical disk, optical 
tape, or machine readable bar code; solid-state electronic storage devices such as 
random access memory (RAM), or read-only memory (ROM); or any other 
physical device or media employed to store a computer program having 

30 instructions for operating one or more computers to practice a method according 



23 



to the present invention. 

The invention has been described in detail with particular reference 
to certain preferred embodiments thereof, but it will be understood that variations 
and modifications can be effected within the spirit and scope of the invention. 
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